A semiconductor memory device having a three dimensional structure comprises an integrated structure of a memory cell array including a plurality of memory cells and a peripheral circuit. The memory cell array includes a stacked body that includes a plurality of electrode layer each stacked via an insulating layer. Memory holes are formed in the stacked body, and the memory cells are provided in the memory holes. The stacked body has an end portion formed into stairs, and each of the plurality of electrode layers is electrically extracted outward through the end portion. The end portion formed into stairs extends around the stacked body, making a chip surface enlarged. Thus, it is desired to suppress such an enlargement of the chip surface.